The Next Bechdel Test

Tests that look behind the camera

Movies don’t just emerge fully formed from the ether; they’re the result of the labor of hundreds and often thousands of people. Most of those people are men. For example: The Annenberg Inclusion Initiative looked at 900 movies released over nine of the past 10 years⁴ and found that only 34 women had worked as directors on those films. If women aren’t in key creative roles — say, if they’re being drummed out of the industry by pervasive, top-to-bottom sexism — then it’s not surprising that the resulting work is skewed. In other words, for every gratuitous sexualizing shot of a woman in a film, there is (almost always) a guy deciding to shoot it that way.

So these three tests look at who makes the movies. Kate Rees Davies, a director, wanted to make sure that not only was there a woman in each and every department, but also that there was a second woman to back the first one up. Rory Uphold, an actress, wanted to see who was on set during filming, to see who was in the room when decisions got made. Jen White, a cinematographer, has developed a tiered metric for judging the gender balance of a crew.⁵

These tests that look at the behind-the-scenes crew are among the least flattering for Hollywood. And, in fact, the overwhelming maleness of film crews was repeatedly cited by most of the women interviewed for this story as a key cause of the issues they’ve experienced in the industry.

"To be honest, there’s no excuse for these kinds of numbers,” said Uphold. “You want to stop sexual harassment in the workplace? Hire more women. End of story."

The Uphold Test

Rory Uphold: writer and actress on “This Is Why We’re Single”

A movie passes if:

The on-set crew is 50 percent women

The Conjuring 2 By the numbersUsing names to estimate gender, “The Conjuring 2” was one of the worst offenders, with men accounting for about 90 percent of the crew.

Show test notes

“On-set crew” refers to directors, second-unit directors and assistant directors, as well as people working in the camera, electrical, cinematography, production management and sound departments. We used Genderize.io to calculate the probability that the first name of all qualifying crew members belonged to a man, counting as a man any crew member whose name belongs to a man more than 90 percent of the time. We also performed a secondary check: We summed up the probabilities that each crew member was male based on their first name and divided that by the total number of crew. Since both those checks estimated that men accounted for more than 50 percent of every crew, all 50 movies failed. In fact, even the best-performing films — “Arrival” and “Moana” — had on-set crews that we estimated to be over 60 percent male. For these movies to get a “pass,” one in six of the people with names that are almost always masculine would have to be women.

The Rees Davies Test

Kate Rees Davies: director and producer

A movie passes if:

Every department has two or more women

Batman v Superman: Dawn of Justice By the numbersEven though the movie passed this test, men made up an estimated 73 percent of the crew of “Batman v Superman: Dawn of Justice.”

Show test notes

We checked each department of each film to see if there were at least two people who had names that belonged to women 90 percent of the time or more. If a department failed that test, we checked to see if at least two generally-but-not-definitely female names or two gender-ambiguous names appeared. And if a department failed that test, we checked to see if all names in the department were most likely male (names that belonged to men at least half the time). Films for which all departments passed the first check passed the Rees Davies test. Films where at least one department failed the third check failed the test. For the leftover films with departments that passed the middle check — and when that department was decisive in whether a film passed or failed the test — we went through the people in that department individually, looking on IMDb, LinkedIn and personal sites to ascertain if two women worked there.

The White Test

Jen White: director of photography; International Cinematographers Guild member

A movie passes if:

Half of the department heads are women

Half the members of each department are women

And half the crew members are women

Don’t Breathe By the numbersWhile still failing, “Don’t Breathe” approached parity, with men making up around 54 percent of the crew.

Show test notes

For the White Test, a department head was defined as the person who could win the Academy Award for the position — essentially anyone billed as a production designer, set designer, art director, special effects supervisor, visual effects supervisor, editor, stunt coordinator, animation director or director of photography. Plus, in the sound category, the production sound mixer, the re-recording mixer and the sound editor. The costume, casting, hair and makeup departments were excluded. Departmentally, this was determined using the same method we used for the Rees Davies Test, but with a 50 percent threshold rather than a two-woman threshold. For movies that can’t pass the full test above, White also gave us standards for a “minimal pass” and a “moderate pass.” Movies get a minimal pass if they have one woman in each department and one woman serving as the head of a department other than casting, costumes, hair and makeup. For perspective, 23 out of 50 films passed the minimal version of the White Test. The moderate pass called for 30 percent of departmental heads and 30 percent of each department’s crew members to be women, but no films cleared those thresholds.

Tests that look beyond white women

When women do get a break in Hollywood, they tend to be white. One study found that among the female leads and co-leads in the top 100 films⁶ of 2016, only three were members of an underrepresented racial or ethnic group. And that figure was identical to 2015’s. Young girls of color have a difficult time finding movie characters who look like them.

“I feel like we all deserve that,” said Lena Waithe, the actress and Emmy Award-winning writer. “Everybody deserves to see positive — not positive, I’m not a big fan of that word — but accurate and layered and complex images of themselves.”

These tests ask if films incorporate women of color at any level. In writer and actress Naomi Ko’s test, it’s as simple as a looking for a non-white, English-speaking woman who talks in five or more scenes. In Waithe’s test, it’s about a balanced portrayal. About 18 percent of the country is Hispanic or Latino, but Hollywood failed a simple test, from producer and writer Ligiah Villalobos, that looks for Latinas in film.

“It is incredibly disappointing,” said Villalobos, “that we are at the end of 2017, that this country has over 50 million Latinos, and that these are the numbers relating to Latina characters in films.”

The Waithe Test

Lena Waithe: Emmy Award-winning writer on “Master of None”

A movie passes if:

There’s a black woman in the work

Who’s in a position of power

And she’s in a healthy relationship

Ghostbusters The Peanut Gallery says:“Why did it have to be the black character who wasn’t the scientist?”

Show test notes

For animated films, we checked IMDb and counted a character as a black woman if she was voiced by a black woman. We counted a position of power as one in which a person exerted control over others or had at least one subordinate.

The Ko Test

Naomi Ko: writer; actress in “Dear White People”

A movie passes if:

There’s a non-white, female-identifying person in the film

Who speaks in five or more scenes

And speaks English

Zootopia The Peanut Gallery says:“Mrs. Otterton is played by Octavia Spencer, so there is a non-white woman in the film. On the other hand, she speaks in two scenes.”

Show test notes

Speaking means dialogue, not fight grunts. For animated films, we checked IMDb and counted a character as non-white if she was voiced by a non-white actor. Originally, the test didn’t specify that the character had to speak English but rather asked if she spoke “in the language of the release,” but all the movies we watched were English-language anyway.

The Villalobos Test

Ligiah Villalobos: producer; head writer of “Go, Diego, Go!”

A movie passes if:

The film has a Latina lead

And the lead or another Latina character is shown as professional or college educated, speaks in unaccented English, and is not sexualized

Zootopia The Peanut Gallery says:“‘Zootopia,’ has Shakira, but her character is sexualized. How do you sexualize a deer?”

Show test notes

We consulted IMDb to see if an actor identified as Hispanic or Latina in her bio. In animated films, we looked at the voice actors.

Tests that look at female protagonists

These three tests — from director Kimberly Peirce, producer Lindsey Villarreal and writer Noga Landau — look at women as characters who have needs and desires and who take actions stemming from those desires over the course of the film. (You know, they act like real people.) A surprising number of films fail to do even that much basic character development work with women. Often, women are reduced to stereotypes or tropes as soon as they’re introduced and then don’t get developed any further. And female characters frequently serve little purpose beyond causing plot problems for male protagonists, or having a baby with a male protagonist, or dying to raise the stakes for a male protagonist.

The Peirce Test

Kimberly Peirce: director of “Boys Don’t Cry,” “Stop-Loss” and “Carrie”

A movie passes if:

There’s a female character who is a protagonist or antagonist with her own story

The female lead has dimension and exists authentically with needs and desires that she pursues through dramatic action

And the audience can empathize with or understand the female lead’s desires and actions

Pete’s Dragon The Peanut Gallery says:“It’s Pete’s dragon, not Grace’s dragon.”

The Villarreal Test

Lindsey Villarreal: producer; assistant on “Mad Men,” “The Strain” and “Bates Motel”

A movie fails if:

A lead female character is introduced as one of three common stereotypes in her first scene: as sexualized; as hardened, expressionless or soulless; or as a matriarch (tired, older or overworked)

But a failing movie can redeem itself and pass if the lead female character is later shown to be three or more of the following:

Someone with a career where she is in a position of authority or power

A mother

Someone who’s reckless or makes bad decisions

Someone who is sexual or chooses a sexual identity of her own

The Secret Life of Pets The Peanut Gallery says:“I thought the cat was introduced as hardened, expressionless and soulless.”

The Landau Test

Noga Landau: writer for “The Magicians”

A movie fails if:

A primary female character ends up dead

A primary female character ends up pregnant

Or a primary female character causes a plot problem for a male protagonist

Deadpool The Peanut Gallery says:“Vanessa causes a plot problem for Deadpool: She’s hot and he’s embarrassed about not being hot.”

Show test notes

After seeing the results, Landau suggested that “male protagonist” be amended to “male hero” to exclude male antiheroes.

Tests that look at the supporting cast

The Annenberg Inclusion Initiative looked at every speaking character in the 100 top-grossing films of 2016. Only 31.4 percent of the 39,788 speaking characters were women. One reason for that: The majority of supporting roles, one-scene parts and bits as extras go to men. Does your protagonist report a crime to a cop? Cop’s probably a man. Do they get stitched up? Doctor’s probably a man. Their president’s probably a man, and the soldiers that he commands are likely men, too. Kate Hagen first picked up on this while watching a lecture by Geena Davis.

“I had never considered how strange it was that most one-scene roles or small parts in film end up going to men by default,” said Hagen, the director of community at The Black List.⁷ “You can never un-see it. The lack of women starts driving you mad. The same goes for crowd scenes: If we're in New York City, why is the crowd 70 percent male and 80 percent white?”

It may be easier for these oversights to slip through when women aren’t involved in key, high-level creative positions during the production of a film. These three tests — from Hagen, two of our movie watchers, and director Rachel Feldman — look past the top-billed actors and study the industry’s systemic casting issues by examining supporting casts and the background players who populate the world that the stars inhabit.

The Hagen Test

Kate Hagen: director of community at the scriptwriter group The Black List

A movie passes if:

Half of one-scene roles go to women

And the first crowd scene features at least 50 percent women

Hacksaw Ridge The Peanut Gallery says:“First crowd scene is zero percent women.”

Show test notes

One-scene characters were counted up by each watcher and scrutinized for gender. To accommodate mildly different definitions of “one-scene character,” each movie-watching pair reconciled their list of one-scene roles to determine whether a movie passed or failed. In animated films, the gender that IMDb listed for the voice actor was considered when other ways of determining gender failed. A crowd was defined as any group in excess of 10 people. For films with no crowds and films where it was impossible to determine the gender of any crowds, that requirement was excluded and the movie passed or failed solely on the percentage of one-scene roles that went to women.

The Koeze-Dottle Test

Rachael Dottle and Ella Koeze: journalists — we helped do this project

A movie passes if:

The supporting cast is 50 percent women

The Magnificent Seven The Peanut Gallery says:“There were so many dusty men in this movie — too many to count.”

Show test notes

Two of our frustrated movie-watchers came up with this test during the research process. To conduct it, we pulled the gender of all actors billed as “supporting” — that is, not lead, lead ensemble or cameo parts — on OpusData, and counted up the gender breakdown among the highest-billed roles (either the half of the supporting cast that had the highest billing or the 10 highest-billed supporting roles, whichever number is lower).

The Feldman Score

Rachel Feldman: director; former chair of the Directors Guild of America’s Women’s Steering Committee

A movie passes with a score of five or higher:

2 points for a female writer or director

1 point for a female composer or director of photography

1 point for three female producers or three female department heads

1 point for a crew that’s 50 percent women

2 points if there’s a female protagonist who determines story outcomes

2 points if no female characters were victimized, stereotyped or sexualized

And 1 point if a sex scene shows foreplay before consummation, or if the female characters initiate or reciprocate sexual advances

The Angry Birds Movie The Peanut Gallery says:“If she had had sex, that would have been the least offensive thing about this movie.”

Show test notes

Writers, directors, composers, producers and department heads were pulled from IMDb. A department head was defined as the person who could win the Academy Award for the position. Only those credited as “producer” — not “executive producer” or other adjective-modified producer — were counted toward the female producer quota.